Papers with uncertainty-aware evaluation method
uBLEU: Uncertainty-Aware Automatic Evaluation Method for Open-Domain Dialogue Systems (2020.acl-srw)
Copied to clipboard
| Challenge: | Existing evaluation metrics for text generation tasks do not consider uncertain responses without writing additional reference responses by hand. |
| Approach: | They propose a human-aided, uncertainty-aware evaluation method for open-domain dialogue systems, BLEU. |
| Outcome: | The proposed method is comparable to existing methods on Twitter and improves state-of-the-art evaluation method RUBER. |